Communication Streams

On the previous page, we introduced the concept of communication streams between services. They form a powerful abstraction that allows service developers to just read and write data to services that are guaranteed to be available, without the need to set up any sockets or bother with low-level interfaces. Though, we did not cover one very important concept yet: how can services understand the messages that are being sent and received? This is especially important in communication between two services that are written in different languages.

Elias Groot

Software Lead, Course Organizer

Before you continue reading, it is good to realize that this tutorial describes our recommended approach to inter-service communication. If you decide to follow along, you can read/write with the services that we offer you and you will enjoy tightly integrated debugging functionalities. Though if you want to use your own (de)serialization logic and byte format, feel free to skip this section.

A Fast, Efficient and Language-Agnostic Wire Format

To allow for real-time communication between multiple services that are possibly written in a different language, they need to agree on a wire format. How should service A serialize its message, so that service B can read and understand it?

Many options are available, but for services that send messages many times per second like on the Rover, the wire-format must be fast (i.e. serialization and deserialization should not incur significant latency) and messages must be encoded efficiently (so that as few bytes as possible need to be transferred). Additionally, the mechanism should be language agnostic: C developers like their data to be encoded as a struct, while Python developers might prefer a native class object.

One wire format that ticks all these boxes is the Protocol Buffer, or protobuf, format. Unless you want to adapt your own communication, there is no need to explore its underlying implementation in-depth. You do need to know that data is encoded to an efficient binary format (unfortunately not human readable) and that you can define the type of messages that can be exchanged in proto files.

These proto files can in turn be compiled or transpiled to a wide variety of languages using the protoc compiler. After compilation, you can just write the messages in the common format of your language of choice, being able to encode it to a byte stream with a single function call.

rovercom: Useful proto Files

Later, we will go over the practical implementation of protobuf for your language of choice. For now it is enough to know that we have defined many useful messages in our rovercom collection. You can find popular definitions here. All official ASE services expect to read and write messages based on these definitions so you need to use them if you want to interface with our services. Additionally, we have tightly integrated our debugging suite with these definitions, so if you decide to let your service output rovercom messages as well, you can enjoy advanced insights as we will see in a later tutorial.